Dirty Statistical Models

We provide a unified framework for the high-dimensional analysis of“superposition-structured” or “dirty” statistical models: where the model param-eters are a superposition of structurally constrained parameters. We allow for anynumber and types of structures, and any statistical model. We consider the gen-eral class of M -estimators that minimize the sum of any loss function, and aninstance of what we call a “hybrid” regularization, that is the infimal convolutionof weighted regularization functions, one for each structural component. We pro-vide corollaries showcasing our unified framework for varied statistical modelssuch as linear regression, multiple regression and principal component analysis,over varied superposition structures.